Convex Optimization Overview

Principles

Definition

Convex optimization is a subfield of mathematical optimization that focuses on the theory and algorithms for minimizing convex functions or maximizing concave functions over convex sets. This academic concept investigates problems where any local optimum is also a global optimum, a property that makes these problems significantly more tractable and efficiently solvable compared to general non-convex optimization problems. Its significance lies in providing a powerful framework with provable guarantees for solving a wide range of problems across engineering, economics, statistics, and computer science.

Ontological type

Core Methods

Theoretical Foundations

Applications

Key developments

Key Figures

Foundations and Unification era

J. A. Nelder [1], associated with the University of Cambridge [3] and Imperial College London [4] during this era, helped shape foundational ideas in convex optimization. His 1965 paper A Simplex Method for Function Minimization [7] introduced the simplex method for function minimization, a practical optimization tool that informed subsequent systematic approaches central to this era's unification efforts. Dimitri P. Bertsekas [2], with ties to the Massachusetts Institute of Technology [5] and Stanford University [6], contributed to maximal monotone operator theory through the Douglas–Rachford splitting method and the proximal point algorithm [8]. The 1992 paper On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators [8] helped unify and stabilize the convergence analysis of operator-splitting methods, shaping the analytic language that underpins interior-point, proximal-point, and conic approaches in this era.

Conic Scaling and Splitting era

Stephen Boyd [1], associated with Harvard University [3] and Stanford University [4], helped shape conic scaling and splitting methods in this era. In 2010, the paper Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers [7] introduced a modular, scalable splitting approach that decomposed large conic problems into simpler subproblems, enabling distributed optimization at scale. J.F. Sturm [2], with affiliations at Erasmus University Rotterdam [5] and Maastricht University [6], contributed to practical conic-optimization tooling in this era. Their 1999 paper Using SeDuMi 1.02, A Matlab toolbox for optimization over symmetric cones [8] made symmetric-cone optimization accessible, enabling interior-point methods for semidefinite and second-order cone programs and accelerating adoption in engineering and signal processing.

Geometry-Aware First-Order era

Mingyi Hong [1], affiliated with the University of Minnesota [3] and the Chinese University of Hong Kong [4], stands as a key figure in the Geometry-Aware First-Order era. Mingyi Hong [1] contributed the 2013 paper A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization [7], which advanced convergence guarantees for nonsmooth, blockwise minimization and underscored the role of geometry-aware first-order techniques in this era. Yurii Nesterov [2], affiliated with the University of Cambridge [5] and KU Leuven [6], authored the 2018 Relatively Smooth Convex Optimization by First-Order Methods, and Applications [8], introducing relatively smooth convex optimization that broadens the applicability of first-order methods beyond Euclidean settings. This framework enabled optimization methods to adapt to non-Euclidean geometries and information-constrained environments, supporting the era's development of two-point and zero-order analyses, distributed schemes, and broader bandit-style optimization.